Enterprises, such as retailers, have hundreds, if not thousands of actions that users may perform via the user interface of the enterprise's applications executing on the users' mobile devices. For example, for a retailer that sells products as well as provides pharmacy services, re-filling a prescription is just one of many different actions or tasks that a user can perform via the enterprise's mobile application. For instance, a user may also update their notification preferences, update their name or other user profile information, clip coupons, check inventory for 70,000+ products, add products to a shopping cart, read millions of reviews, etc. Ensuring that all of these types of user experiences execute as desired and/or as defined via the user interface of an enterprise's mobile application and ensuring that particular user actions do not break or negatively impact other functionality of the mobile application is no small task, as thousands of test cases may be needed to sufficiently test the user interface of an enterprise's mobile application. For example, a test version of the enterprise's mobile application may be executed (e.g., on a physical mobile device and/or via a virtual machine), and a sequence of user actions specified in a test case may be automatically performed on the executing test of the mobile application, e.g., “tap on ‘Pharmacy’ user control, then wait 3 seconds, then tap on ‘Refill from Account’ user control, then enter {userID} in the input field called “username . . . ,” and so on. Each user interface test case is designed to ensure that a particular piece of functionality is working as intended and/or that certain sequences of user actions do not negatively impact other functionality of the mobile application.
Writing test cases is time consuming and difficult, and ensuring that a test case is written in a way that reflects how a user would actually navigate an enterprise's mobile application in the real world is even more difficult.
Currently known techniques for automatic generation of user interface test cases include training a machine learning model by using crowd-sourced data, e.g., by using user interface action and sequencing data that has been obtained from “a crowd” of mobile applications executing at mobile devices, and then utilizing the trained model to generate unit test cases. However, as such techniques use classifying and other statistical, aggregate data analysis techniques to generate the new test cases, the accuracy of at least some of such test cases with respect to the actions that real-world users actually perform may be inaccurate, as such test cases are synthetic, i.e., such test cases are new test cases that have been generated by the model. For example, when various types of inputs are omitted from or not indicated within the training data (e.g., geographic location, user demographic, etc.), the impact of such inputs may accurately reflect real-world behavior in test cases generated by the model. Further, modeling techniques may generate, in theory, an infinite number of test cases, making it difficult to ascertain which are the most useful test cases that correspond to actual, real-world user actions as, in practice, the amount of resources an enterprise has available for testing mobile applications (e.g., computer processing resources, human resources, time, money, etc.) is limited, and the enterprise would want to perform only the test cases that have the most impact on the quality of their mobile application user interface given their limited resources. Similar drawbacks also pertain to automated test generation techniques that utilize models that are created using techniques other than training models, e.g., natural language processing, finite state machine generation, class diagrams and constraints, a priori defined specifications, etc.
Other currently known automated test generation techniques include utilizing randomly generated sequences of user inputs for test cases, utilizing combinatorial testing techniques on sequences of user inputs and/or domains of user inputs, generating UI graphs and traversing paths therein, automatically generating test cases from exploring the various branches of a mobile application's source code, and the like. However, such techniques also suffer from accurately representing actual, real-world user actions, and/or from being able to ascertain the most useful and beneficial test cases.